Papers by Vinayak K Doifode
LatentGate: Low-Latency Semantic Routing via Frozen-Backbone Probing of Small Language Models (2026.acl-industry)
Copied to clipboard
| Challenge: | Prompt-based LLM routers deliver strong semantic reasoning but incur prohibitive latency and cost that scales with agent count. |
| Approach: | They propose a non-generative router that extracts mean-pooled hidden states from a frozen small language model and trains a lightweight linear probe for agent classification. |
| Outcome: | The proposed router achieves 98.8% in-domain and 80.0% OOD accuracy across 5 SLM backbones and 100 enterprise agents. |